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SUMMARY 


Two  training  concepts  designed  to  improve  A7  Category  I pilot  training 
effectiveness  were  developed  and  tested.  One  concept  involves  the  reduction 
of  recycle  training  through  the  analysis  of  night  FCLP  landing  performance, 
diagnostic  training  feedback  and  NCLT  remedial  instruction  for  all  novice 
pilots.  A second  concept  concerns  the  fleet  follow-up  and  performance  feed- 
back for  all  recycled  replacement  pilots  (RPs). 

The  following  answers  were  obtained  for  four  basic  research  questions 
asked  about  recycled  pilots. 

Question  #1;  Can  potential  recycle  pilots  be  identified  early? 

Answer:  Potential  recycle  pilots  can  be  identified  by  the  fifth  night 
FCLP  period  by  selecting  an  FCLP  cutting  score  and  determining  if  the  RP 
is  either  erratic  in  performance  or  below  the  cutting  score  in  three  or  four 
of  the  first  five  night  FCLP  periods. 

Question  #Z:  Can  specific  landing  problem  that  lead  to  recycle  training  be 
identified? 

Answer:  An  analysis  of  FCLP  Landing  Signal  Officer  (LSO)  comments 
can  be  used  to  evaluate  each  RP  landing  technique  and  provide  diagnostic 
feedback  on  landing  deficiencies  using  a landing  trend  analysis  form  to  sum- 
marize the  data. 

Question  #3:  Are  landing  problems  correctable  without  recycle? 

Answer:  This  question  requires  additional  study  before  a firm  answer 
is  stated.  In  this  study  the  landing  problems  of  one  RP  who  was  a low  per- 
former were  sufficiently  corrected  using  NCLT  remedial  training  to  allow 
him  to  pass  carrier  qualification  (CQ)  on  his  initial  try  and  avoid  recycle 
training.  A second  predicted  recycle  RP  was  not  salvaged  by  remedial 
training  and  was  recycled  through  a complete  second  training  cycle  for  CQ. 

Question  #4:  Does  CQ  performance  improve? 

Answer:  The  results  clearly  indicate  an  improvement  in  Nugget  CQ 
performance  compared  with  non-remedial  trainees.  An  exceptionally  high 
night  boarding  rate  of  83  percent  was  obtained  compared  to  an  average  night 
boarding  rate  of  62  percent.  Whether  the  imprbved  performance  is  sus- 
tained in  the  fleet  should  be  foUowed-up  in  a longitudinal  study  of  all 
Category  I pilots. 

A fleet  disqualification  rate  of  37.  5 percent  for  eight  recycled  pilots 
was  obtained  when  information  on  their  fleet  performance  was  analyzed. 
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PREFACE 

The  Human  Factors  Laboratory  of  the  Naval  Training  Equipment  Center 
has  been  engaging  in  a continuing  program  of  research  with  three  general 
goals:  1)  improvement  of  the  methodology  associated  with  research  in  train- 
ing effectiveness  evaluations,  2)  improvement  of  training  methods  with  its 
corollary  benefit  of  making  more  effective  use  of  simulators,  and  3)  improve- 
ment of  future  simulator  design  and  use  through  application  of  research 
results.  Previous  efforts  have  brought  into  focus  the  contribution  that 
objective  measures  make  in  evaluation  and  the  importance  of  proper  simula- 
tor usage  on  transfer  of  training.  In  a recent  evaluation  of  the  NCLT,  using 
an  experimental  syllabus,  a positive  transfer  of  training  from  the  NCLT  to 
actual  A7E  carrier  qualification  was  found. 

The  present  study  is  an  exploratory  effort  in  training  methods  improve- 
ment. Results  indicate  that:  1)  an  analysis  technique  can  be  applied  to  FCLP 
performance,  2)  the  results  of  the  analysis  provide  a diagnosis  of  landing 
performance,  3)  diagnostic  information  can  be  translated  into  remedial  train- 
ing objectives  that  can  be  accomplished  in  the  NCLT,  4)  NCLT  remedial 
training  produces  an  improvement  in  landing  performance  during  FCLP, 

5)  the  experimental  NCLT  training  provides  positive  transfer  of  training  from 
FCLP  to  actual  carrier  qualification,  6)  the  procedure  is  feasible  for  on-going 
operational  training  for  carrier  qualification,  and  7)  no  change  in  trainer  hard- 
ware is  required  to  accomplish  the  increase  in  training  effectiveness. 

An  important  aspect  of  the  study  is  that  a method  to  determine  specific 
and  individual  remedial  NCLT  instructional  objectives  has  been  integrated 
with  the  established  training  program  objectives.  Tailoring  of  any  aspect  of 
flying  training  to  the  individual  is  often  sought  but  seldom  realized.  This 
study  provides  the  means  to  individualize  instruction  in  the  NCLT.  The  con- 
cept can  be  applied  across  other  weapon  systems  that  use  visual  landing 
display  simulators.  Similar  concepts  can  also  be  developed  for  application 
in  other  types  of  simulators. 

The  study  discussed  in  this  report  was  made  possible  through  the 
cooperation  and  interest  of  VA-174  whose  Commanding  Officer,  CDR  J. 
McCain,  gave  us  the  necessary  support  and  liaison.  Outstanding  contributions 
to  the  formulation  of  the  project  goals  and  their  implementation  were  made  by 
the  team  of  LSOs  assigned  to  the  Fleet  Readiness  Squadron.  The  primary 
impetus  was  provided  by  LT  Ross  Fisher  whose  interest  and  enthusiasm  led 
to  the  field  test  where  he  was  ably  assisted  by  LT  Ron  Boraten  and  LT  Mike 
Wilson.  Their  extraordinary  assistance  and  additional  work  made  the  project 
that  much  easier  to  accomplish. 

Pete  Pettigrew  of  Dunlap  and  Associates'  staff  provided  a broad  critique 
and  review  of  the  material  and  provided  several  insightful  comments  especially 
in  potential  applications  of  the  results  to  fleet  training.  His  unique  blend  of 
operational  and  project  experience  clarified  many  of  the  concepts  presented 
and  his  overall  involvement  with  the  project  was  invaluable. 

WILLIAM  B.  BONEY  / 

Scientific  Officer 
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SECTION  I 
BACKGROUND 


In  July  1976  a transfer  of  training  effectiveness  study  for  the  Night 
Carrier  Landing  Trainer  (NCLT)  was  completed.  The  results  clearly 
indicated  the  positive  transfer  of  training  from  the  NCLT  to  actual  Carrier 
Qualification  (CQ)  trials  for  novice  (Category  I)  Replacement  Pilots  (RPs). 
Fifteen  months  of  data  collection  were  required  to  obtain  a minimum  sample 
of  pilots  for  the  trainer  (N=26)  and  the  control  (N=27)  groups  which  provided 
ample  time  to  observe  the  A7  Fleet  Replacement  Squadron  (FRS)  training 
process. 

One  of  the  more  interesting  research  results  concerned  the  additional 
time  and  flight  hours  required  to  qualify  pilots  who  failed  carrier  landing 
training  in  their  initial  training  cycle.  Pilot  trainees  were  disqualified 
either  during  Field  Carrier  Landing  Practice  (FCLP)  or  CQ.  Such  pilots 
were  usually  recycled  through  another  complete  training  period  which,  on 
the  average,  resulted  in  12  more  weeks  at  the  FRS  and  19  percent  more 
flight  hours  than  those  pilots  who  successfully  completed  training  in  their 
first  attempt. 

"The  problem  with  recycle  and  additional  training  time  is  cost. 
Increased  time  at  the  Fleet  Replacement  Squadron  (FRS)  (+12  weeks)  and 
more  A7E  flight  time  (+19  percent)  means  delay  in  reporting  to  a fleet 
squadron  and  more  money  for  fuel.  Time  and  cost  are  also  increased  due 
to  additional  CQ  ship  trials,  LSO  time  expenditure  and  aircraft  demand  to 
meet  the  burden  of  more  FCLP  and  CQ  trials.  ”1 

Based  on  our  observations  of  FRS  training  it  seemed  highly  probable 
that  new  training  concepts  which  address  the  problem  of  recycle  training 
time  and  increased  flight  hours  would  improve  FRS  training  effectiveness. 
One  proposed  concept  is  that  of  individualized  diagnostic  training  feedback 
and  remedial  instruction  for  Category  I novice  pilots. 


^Brictson,  C.A.  and  Burger,  W.  J.  Transfer  of  Training  Effectiveness: 
A7E  Night  Carrier  Landing  Trainer  (NCLT),  Device  2F103. 
NAVTRAEQUIPCEN  74-C-0079-1,  August  1976. 
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Another  interesting  aspect  of  our  observations  had  to  do  with  the  ' 

eventual  fleet  performance  of  recycled  Category  I pilots.  How  did  repeaters 
do  in  the  fleet?  While  they  represent  a considerable  investment  in  training 
by  the  time  they  complete  FRS  instruction  very  little  formal  follow-up  of 
their  fleet  performance  is  conducted.  A second  proposed  training  concept 
is  performance  feedback  from  fleet  squadrons.  One  of  the  topics  covered 
later  in  the  report  is  an  informal  fleet  follow-up  of  a small  number  of  A7E 
pilots  who  were  recycled  in  their  CQ  training. 

There  are  four  crucial  training  questions  about  recycled  pilots  that 
are  emphasized  in  this  report.  They  are: 

1.  How  can  they  be  identified  early  in  the  FRS  training  program? 

2.  What  are  the  specific  landing  problems  that  lead  to  their 
recycle  training? 

3.  How  can  their  landing  problems  be  corrected? 

4.  How  can  their  CQ  performance  be  improved? 

This  report  provides  a description  of  the  methods  used  to  obtain  preliminary  v 

answers  to  these  questions  which  are  important  to  A7  FRS  training  effective- 
ness, expeditious  squadron  assignment,  and,  ultimately,  fleet  performance. 

The  following  material  reviews  the  scope  of  carrier  landing  training 
as  well  as  the  nature  of  recycle  training  and  considers  the  potential  impact 
of  these  two  training  concepts. 
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SECTION  II 

CARRIER  LANDING  PERFORMANCE  TRAINING 


An  evaluation  of  FRS  CQ  training  was  completed  in  order  to  determine 
how  low  performing  RPs  can  be  identified  early  in  the  training  cycle.  The 
ground  and  flight  training  syllabus  for  Phase  III  is  shown  in  Table  1.  An 
analysis  of  Phase  III  training  shows  that  four  types  of  training  take  place  in 
an  effort  to  teach  sufficient  knowledge  and  skills  to  the  RP  so  that  he  can 
successfully  demonstrate  his  landing  ability  aboard  ship.  This  training  is 
essentially  sequential  in  nature  with  some  overlap  provided  between  NCLT 
and  FCLP.  Training  is  listed  in  Table  2 in  decending  order  of  importance. 
CQ  performance  is  obviously  most  important  to  RP  success  in  passing  FRS 
training.  While  a pilot  may  fail  in  either  CQ  or  FCLP,  NCLT  or  lectures 
are  never  failed  but  simply  repeated  to  encourage  greater  proficiency. 
Probably  95  percent  of  all  performance  related  failures  during  FRS  training 
occur  during  FCLP  and  CQ. 


TABLE  2.  PHASE  HI  LANDING  PERFORMANCE  TRAINING 


Training  Method 

Output 

1. 

Lectures 

Landing  information  and  procedures. 

2. 

Night  Carrier  Landing 
Trainer  (NCLT) 

Practice  landings  with  simulated 
aircraft  and  carrier. 

3. 

Field  Carrier  Landing 
Practice  (FCLP) 

Practice  landings  with  real  aircraft, 
stationary  airfield. 

4. 

Carrier  Chialification  (CQ) 

Actual  landings  with  real  aircraft, 
moving  carrier. 

( 
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TABLE  1.  PHASE  lU  GROUND  & FLIGHT  TRAINING  SYLLABUS  * 


GROUND  training  SUMMARY 

FLIGHT  SUPPORT  PHASE  LECTURES  - LANDING 

FSCQ-1  Landing  Techniques 

FSCQ-2  Day  FCLP/Optical  Landing  Aids 


Classroom 

Hours 


1.0 

1.5 

2. 5 hrs. 


CARRIER  QUALIFICATION  LECTURES 


FSCQ-3 

Day  FCLP  Review 

1.  0 

FSCQ-4 

Night  FCLP/CCA  Procedures 

1.5 

FSCQ-5 

NCLT  Brief/ LSO  CV  NATOPS 

1.  0 

FSCQ-6 

Shipboard  Orientation 

1.  0 

FSCQ-7 

Enroute/Recovery  Procedures 

1.  5 

FSCQ-8 

Day  CQ 

2.0 

FSCQ-9 

Night  CQ 

1.  5 

FSCQ-10 

Review  and  Exam 

1.  0 

10.  5 hrs. 

total 

13. 0 hrs. 

FLIGHT  TRAINING  SUMMARY  Flight  Hours 


MLP  - (Mirror  Landing  Practice) 

PP-1 

PP-2 


1.  0 
1.  0 

2. 0 hrs. 


FCLP  - (Field  Carrier  Landing  Practice) 


PP3-6  Day  FCLP  4. 0 

PP7-15  Night  FCLP  9.  0 

13.0  hrs. 

CQ  - (Carrier  Qualification) 

PQl-2  Day  CQ  4.0 

PQ3-4  Night  CQ  3. 0 

7.  0 hrs. 

total  22.  0 hrs. 
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Lectures.  Landing  lectures  supply  the  necessary  information  about 
aircraft  and  ship  systems  related  to  landing  and  generally  prepare  pilot 
trainees  for  day  and  night  FCLP,  NCLT  and  eventually  CQ.  However, 
since  no  attrition  occurs  here  there  is  no  indication  yet  of  how  well  the 
pilot  performs  the  landing  task,  merely  some  feedback  (exams,  questions, 
interest,  etc.  ) on  pilot  intake  and  understanding  of  pertinent  landing  infor- 
mation and  procedures. 

NCLT.  NCLT  provides  the  first  opportunity  to  observe  the  training 
behavior  of  the  RP.  During  the  NCLT  study  a standard  NCLT  syllabus 
consisting  of  6.  5 hours  of  simulator  training  was  developed  by  senior  LSOs. 
Table  3 describes  the  NCLT  training  schedule.  Extensive  use  is  made  of 
the  final  approach  simulation  capability  of  the  NCLT.  About  85  simulated 
night  carrier  approaches  are  sched\iled  for  each  RP  on  the  NCLT  (during 
the  NCLT  study  the  average  was  80  passes  per  pilot).  The  sequence  of 
NCLT  training  provides  for  some  overlap  with  FCLP  training,  usually 
during  the  early  and  middle  portions  of  night  FCLP,  about  one  week  prior 
to  CQ  detachment. 

Realistically,  the  NCLT  has  many  untapped  resources  and  capabil- 
ities. Consider,  for  example,  the  following  description  of  LSO  interface 
with  the  NCLT. 

"The  LSO  instructor  can  monitor  trainee  final  approach  and  vary  the 
degree  of  difficulty  of  each  approach  by  selecting  different  aircraft,  carrier 
or  environmental  (sea  state,  wind,  horizon  intensity,  etc. ) characteristics. 
Fourteen  aircraft  emergencies  can  also  be  programmed  during  any  approach 
trial.  For  monitoring  purposes  the  LSO  can  obtain  records  from  an  x-y 
plotter  for  altitude  and  lateral  error  deviations  from  desired  glideslope  and 
lineup.  Display  freeze  is  possible  during  fli^t  and  a replay  function  allows 
a review  of  the  last  minute  of  final  approach  for  LSO/pilot  debriefing  and 
analysis.  Terminal  approach  conditions  (wire  number,  sink  speed,  etc. ) 
also  can  be  displayed  at  trainee  and  LSO  stations  and,  if  desired,  a hard- 
copy of  the  results  can  be  obtained  from  the  line  printer.  The  trainee  can 
"fly”  the  aircraft  with  complete  freedom  and  is  provided  with  realistic 
aircraft  sounds  throughout  the  flight.  Carrier  arrestment  is  simulated  by 
stopping  the  CRT  display.  In  addition,  the  device  has  the  capability  of 
simulating  bolters,  touch-and-goes,  wave-offs  (and  ramp  strikes)  and 
allows  the  trainee  to  reenter  the  final  approach  pattern  after  each  missed 
landing. 


1 


Op  cit. 


Brictson,  C.A. 
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TABLE  3.  NCLT  STANDARD  SYLLABUS  SCHEDULE* 


Period 

Description 

Average 
Ball  Control 
Approaches 

Time 

(hrs) 

NCLT-1 

1 CCA  from  marshall  to  touch  and  go 

3 CCA  from  4.  2 to  touch  and  go  to 
bolter  pattern 

4 

1.  0 

NCLT-2 

1 CCA  from  marshall  to  touch  and  go 

2 CCA  from  4.  2 to  landing  to  bolter 
pattern 

3 

0.5 

NCLT-3-6 

10  Ball  control  approaches  from  1 to 

1 . 5 to  landing 

Vary  weather  conditions 

40 

2.0 

NCLT-7-10 

4 CCA  from  4.  2 to  landing 

4 Ball  control  approaches  from  1 to 

1 . 5 to  landing 

Vary  weather  and  approach  conditions 

32 

2.  0 

NCLT- 11 

4 CCA  approaches  from 4.  2 to  landing 
to  bolter  pattern 

Vary  weather  and  approach  conditions 
Simulate  emergency  aircraft  conditions 

4 

0.5 

NCLT- 12 

1 CCA  approach  from  marshall  to  landing 
1 CAT  shot  into  bolter  pattern 

Vary  weather  and  approach  conditions 

2 

0.5 

Simulate  emergency  aircraft  conditions 

... 

Total 

85 

6.5 

* Note:  This  syllabus  has  undergone  considerable  change  since  its 

use  in  the  NCLT  study  (1976).  The  trend  is  toward  fewer 
periods,  more  Whitehouse  (FCLP)  passes  early  in  training 
and  fewer  overall  ball  control  approaches.  The  results  of 
these  innovations  are  not  yet  formalized  or  documented. 

t 
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In  other  words,  the  NCLT  has  a training  playback  capability  which 
can  be  used  to  provide  knowledge  of  results  to  the  trainee  including  carrier 
landing  trend  analysis  information.  Additional  performance  measure- 
ment and  scoring  techniques  are  now  under  development.  2 

These  relatively  untapped  capabilities  are  emphasized  here  because 
it  is  precisely  through  these  unique  monitor,  control  and  input  functions 
that  learning  "how  to  land"  can  be  enhanced,  promoted  and  individualized, 

FCLiP.  Field  carrier  landing  practice  is  the  first  intensive  landing 
practice  for  the  RP  using  actual  aircraft  and  carrier  landing  aids  coupled 
with  carrier  landing  simulated  by  deck  markings  painted  on  a runway.  The 
LSO  conducts  FCLP  much  the  same  as  he  does  CQ,  by  monitoring  and 
communicating  with  RPs  during  each  day  and  night  pass  about  the  quality 
of  their  final  approach  and  landing,  as  if  it  were  being  made  on  an  aircraft 
carrier.  Preflight  briefs  are  held  and  debriefs  provided  on  each  pass  which 
is  recorded  in  the  LSO  log  book.  Usually  eight  to  twelve  RPs  are  scheduled 
for  each  FCLP  cycle.  Normally,  10  passes  per  period  are  given  to  each  RP. 
LSO  workload  amounts  to  monitoring  and  debriefing  about  100  night  passes  for 
each  of  nine  FCLP  periods.  Close  to  900  passes  are  monitored,  graded, 
summarized  and  averaged  during  night  FCLP,  usually  by  two  LSOs.  Each 
pass  is  graded  and  summed  according  to  a subjective  LSO  grading  scale 
ranging  from  1.  0 to  5.  0.  The  grading  system  is  described  in  Table  4. 


TABLE  4.  LSO  LANDING  GRADING  SYSTEM 


Grade 

Value 

Description 

OK 

5 

Perfect  pass 

OK 

4 

Reasonable  deviations  with 
good  corrections 

(OK) 

3 

Reasonable  deviations 

2 

Below  average  but  safe  pass 

WO 

2 

Waveoff 

OWO 

2 

Own  Waveoff 

Comment 

1 

Emphasized  correction 

C 

0 

Unsafe,  gross  deviations 
inside  waveoff  point 

2 

Klein,  T.  J.  and  Mattlage,  C.E.  Pilot  performance  measurement  system 
for  the  A7  night  carrier  landing  trainer  (NCLT).  Vought  Corp.  Undated, 
1976. 
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To  collect,  organize,  sum  and  average  this  massive  data  bank  on 
landing  performance  the  L.SO  relies  exclusively  on  manual  recording.  The 
administrative  sequence  goes  something  like  this.  Record  and  grade  each 
day  and  night  FCLP  pass  in  LSO  log  (1300  passes).  Sum  grades  for  each 
RP  for  each  period.  Cut  up  each  FCLP  sheet  (3  forms  per  sheet)  and  com- 
pile each  RPs  FCLP  forms  separately.  Sum  day  and  night  FCLP  forms  to 
obtain  overall  grade.  Record  grade  on  Carqual  Phase  Summary  Form. 
Compute  number  of  periods,  approaches  and  landings,  grade  and  class 
average  for  day  and  night  FCLP. 

FCLP  training  usually  covers  from  two-three  weeks  of  time,  depending 
on  weather  and  schedules  with  one  firm  requirement  being  that  the  last  FCLP 
period  usually  not  exceed  five  days  prior  to  CQ. 

CQ.  Carrier  Qualification  (CQ)  training  culminates  the  FRS  training 
cycle.  Successful  completion  of  CQ  is  the  final  performance  test  for  demon- 
strating the  combination  of  knowledge  and  skills  required  for  safe  arrestment 
day  and  night  aboard  an  operational  aircraft  carrier.  The  minimum  require- 
ments for  initial  qualification  of  Category  I pilots  are  two  touch-and-go 
landings,  ten  day  arrested  landings,  and  six  night  arrested  landings. 
Furthermore,  a pilot  may  not  exceed  the  following  operational  limits  in  any 
24-hour  period: 

. 10  arrested  landings 

. 4 night  arrested  landings 

. 6-1/2  hours  in  cockpit 

. 3 flights 

Under  special  situations  the  LSO  may  reconrunend  waivers  of  the  above  to 
twelve  arrested  landings  and  six  night  arrestments. 

The  same  grading  system  used  for  FCLP  is  used  by  LSOs  to  evaluate 
CQ  performance.  In  terms  of  day/night  quals,  a minimum  of  two  day  land- 
ings and  one  catapult  launch  must  be  completed  prior  to  night  qualification. 
The  LSO  is  the  final  decision  maker  in  whether  an  RP  has  satisfactorily 
completed  day  landings  prior  to  any  night  work.  An  LSO  may  also  recom- 
mend additional  landings  if  pilot  performance  justifies  such  an  action.  Time 
required  to  complete  this  training  evolution  varies  from  a minimum  of  two 
days  and  nights  to  as  much  as  one  week  depending  upon  number  of  qualifiers, 
ship  schedule,  weather  and  other  operating  conditions. 

3 

NATOPS  Manxial:  Landing  Signal  Officer.  Dept,  of  the  Navy,  15  Nov.  1975. 
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CQ  Recycle  Training.  A typical  Category  I replacement  pilot  requires 
101.  6 A7  flight  hours  to  complete  the  FRS  training  cycle  successfully.  This 
compares  closely  to  the  standard  number  of  flight  hours  proposed  in  the  A7 
flight  training  syllabus  (101.  3).  In  the  NCLT  transfer  of  training  study  we 
found  that  8 out  of  27  pilots  were  disqualified  during  Phase  III  training  and 
recycled  through  FCLP/CQ.  The  recycle  process  added  19  percent  more 
flight  time  to  the  syllabus,  increasing  it  to  an  average  of  120.  1 hours. 

About  half  of  the  increase  was  in  night  flight  time  which  averaged  38.  3 hours 
compared  to  28.  7 hours  for  successful  RPs. 

Besides  the  considerable  increase  in  flight  time,  each  recycled  RP 
was  delayed  in  reporting  to  a fleet  assignment  by  an  average  of  12  weeks. 
This  delay  was  caused  by  necessity.  Recycled  RPs  must  wait  for  the  next 
training  class  to  repeat  their  FCLP/CQ  periods  and  these  periods  are 
scheduled  as  a function  of  carrier  availability.  Once  a pilot  is  disqualified 
during  either  FCLP  or  CQ  he  is  usually  required  to  drop  back  one  class  to 
repeat  four  hours  of  day  FCLP,  nine  hours  of  night  FCLP  and  seven  hours 
of  day  and  night  CQ.  These  20  hours  of  landing  practice  are  shown  in  Table  5 
and  represent  the  19  percent  increase  in  flight  time  for  recycled  pilots. 


TABLE  5.  NUMBER  OF  PRACTICE  LANDINGS  AND  FLIGHT  HOURS 

BY  TRAINING  PHASE 


CQ  Training 
Phase 

Landings 

Flight  Hours 

Day  FCLP 

40 

4 

Night  FCLP 

90 

9 

Day  CQ 

10 

4 

Night  CQ 

6 

_3 

Totals 

146 

20 

When  a pilot  is  recycled  he  participates  in  additional  training  that 
results  in  as  many  as  130  more  FCLP  landings  in  order  to  pass  success- 
fully to  the  CQ  phase.  At  present,  there  is  little  or  no  individualized 
training  as  far  as  number  of  FCLPs  that  must  be  accomplished  prior  to 
CQ.  All  pilots  repeat  the  entire  FCLP  in  most  cases. 
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To  understand  the  current  carrier  landing  training  for  A7  pilots  it  is 
first  necessary  to  grasp  three  dominant  factors. 

. Recycle  rate 
. Recycle  decision 
. Recycle  resiilts 

First  of  all,  although  recycle  rate  tends  to  fluctuate,  the  generally 
accepted  average  recycle  for  A7  Category  I RPs  (1976)  is  about  26  percent. 
One  in  four  drop  back  one  class.  Usually  the  dropbacks  are  recycled  for 
another  CQ  syllabus.  In  our  NCLT  transfer  of  training  study  the  recycle 
rate  for  RPs  without  NCLT  training  was  30  percent  and  4 percent  for  RPs 
who  were  given  the  standard  80  ball  control  passes  in  the  NCLT  prior  to  CQ. 

A second  factor  in  FRS  training  is  that  almost  all  recycle  occurs  in 
Phase  III  Training.  Very  seldom  is  an  RP  disqualified  during  Phase  I or 
Phase  II  Training.  Thus,  the  LSO  becomes  the  primary  decision  maker 
about  recycle.  He  decides  how  the  RP  performs  in  FCLP,  whether  or  not 
to  take  him  for  ship  CQ  trials,  and  whether  he  passes  day  and  night  CQ. 

All  recycle  training  in  the  NCLT  study  was  identified  either  during 
night  FCLP  or  shipboard  CQ.  Out  of  eight  recycles,  three  failed  night 
FCLP  and  four  failed  night  CQ.  The  lone  recycle  from  the  trainer  group 
failed  during  day  CQ.  The  disqualified  RPs  were  further  broken  down  by 
flight  experience.  All  eight  were  found  to  be  direct  from  the  Training 
Command  (Nuggets)  with  an  average  jet  flight  experience  of  about  330  hours. 
Table  6 summarizes  the  data. 


TABLE  6.  A7E  NCLT  STUDY  DISQUALIFICATION  SUMMARY 


Recycle  Summary 

NCLT  (1) 

No-NCLT  (7) 

Training  Command 

1 

7 

Recycle  percent 

8%  (1/13) 

44%  (7/16) 

Source: 

Day  FCLP 

0 

0 

Day  CQ 

1 

0 

Night  FCLP 

0 

3 

Night  CQ 

0 

4 
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The  third  factor  that  predominates  in  FRS  training  is  that  most  of  the 
recycled  pilots  pass  CQ  on  their  second  trial.  Apparently,  given  the  addi- 
tional flight  time  and  landing  experience  the  recycled  RP  can  successfully 
complete  Phase  III  training  and  qualify  day  and  night  aboard  ship.  Although 
recycling  RPs  is  costly  and  time  consuming  it  appears  to  be  effective  if 
passing  CQ  is  the  criterion.  The  ultimate  criterion,  however,  is  how  well 
pilots  land  during  their  fleet  assignments  and  deployments.  The  relation- 
ship between  CQ  performance  and  fleet  performance  has  not  yet  been  sub- 
jected to  study  but  would  be  useful  to  obtain  training  validation  and 
effectiveness  data  for  the  FRS  program. 
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SECTION  m 

NIGHT  FCLP  PERFORMANCE  ANALYSIS 


As  we  have  indicated,  each  FCLP  practice  landing  is  graded  by  an 
LSO  and  the  grades  for  each  FCLP  period  are  summarized  to  provide  an 
LSO  qualitative  index  of  FCLP  performance.  Class  averages  are  then 
computed  for  day  and  night  FCLP.  Considering  the  average  number  of 
FCLP  periods  per  class  a large  amount  of  training  data  is  collected.  With 
a normal  class  size  of  twelve  RPs  and  each  pilot  receiving  40  day  and  100 
night  FCLPs  there  are  about  1680  practice  landings  per  class.  That  total 
represents  a data  base  that  was  useful  for  providing  information  on  the 
relative  progress  of  each  RP  as  he  proceeded  through  Phase  HI  training. 

For  example,  were  there  any  differences  in  FCLP  scores  for  recycle 
pilots  as  opposed  to  successful  pilots?  What  differences  and  variability 
existed  among  the  classes  and  between  the  pilot  trainees?  Finally,  could 
the  differences  be  used  to  identify  potential  recycle  pilots  based  on  FCLP 
performance  progress  as  reflected  by  LSO  grades? 

To  determine  how  potential  recycle  pilots  can  be  identified  prior  to 
recycle  a look  was  taken  at  how  these  low  performers  do  in  comparison 
with  their  successful  RP  peers.  Again,  using  data  from  the  NCLT  study 
it  was  possible  to  survey  and  summarize  the  FCLP  performance  of  53 
Category  I replacement  pilots.  These  data  consisted  of  day  and  night  final 
approaches  across  successive  FCLP  periods  during  the  last  two  weeks  of 
training  and  just  prior  to  CQ.  Because  recycles  occurred  only  during  night 
FCLP  or  CQ  only  night  data  were  considered  meaningful  for  analysis. 

Those  data  in  cumulative  array  represented  over  5,  000  night  field  carrier 
landings. 

NCLT  vs  No-NCLT.  Figures  1 through  4 depict  those  5,  000  landings. 
Figure  1 illustrates  the  successive  differences  in  average  FCLP  grades  for 
pilots  who  received  NCLT  training  and  those  who  did  not.  For  diagnostic 
reasons,  the  most  significant  differences  occur  starting  at  period  number  5. 
Here,  for  the  first  time,  a difference  between  the  two  groups  emerges. 

This  relative  performance  difference,  as  recorded  by  LSO  grades,  was 
consistently  higher  for  NCLT  pilots  from  period  5 through  period  8.  To 
appreciate  the  differences  in  performance,  recall  that  each  data  point  is 
the  average  for  26  RPs,  each  of  whom  had  about  10  approaches  per  period. 
That  translates,  roughly,  into  an  average  of  260  approaches  per  FCLP 
period  for  both  groups. 
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Figure  1,  Comparison  of  NCLT  vs  No-NCLiT  groups  across  successive  night  FCLP  periods. 
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Landing  performance  differences  between  the  two  groups  were  found 
to  be  statistically  significant  in  two  of  the  nine  periods.  The  average  night 
performance  for  periods  6 and  8 was  significantly  different  at  a probability 
of  p<.  06  and  p<.  05,  respectively.  Since  the  two  groups  were  treated  to 
different  training  protocols,  it  was  concluded  that  NCLT  training  positively 
improved  FCLP  training  as  reflected  by  LSO  grading  techniques. 

Period  9 was  primarily  used  as  a confidence  builder  for  pilot  trainees 
since  it  usually  was  the  last  night  period  prior  to  carrier  qualification. 

LSOs  apparently  used  grading  as  a technique  to  instill  confidence  and  assur- 
ance in  the  RP  about  his  landing  techniques  and  capability.  For  this  reason 
the  sharp  rise  in  LSO  grades  for  period  9 was  treated  as  an  artifact  account- 
able to  LSO  grading  bias. 

Experienced  vs  Inexperienced  RPs.  Experienced  Category  I RPs  were 
operationally  defined  as  those  pilot  trainees  who  had  over  1,  100  jet  flight 
hours  compared  to  just  over  300  hours  for  inexperienced  RPs  or  so-called 
Nuggets.  The  experienced  Category  I pilots  consisted  of  POWs,  Stashes 
and  Plowbacks  (jet  and  prop).  The  inexperienced  RPs  were  newly  designated 
aviators  who  reported  for  FRS  training  direct  from  the  Training  Command. 

Figure  2 compares  experienced  RPs  across  both  trainer  and  no-trainer 
categories.  The  NCLT  group  of  RPs  started  slower  than  their  counterparts, 
but,  again,  by  the  fifth  night  period  began  to  demonstrate  higher  performance 
and  maintained  that  superiority  throughout  the  FCLP  training.  While  better 
performance  was  demonstrated  it  was  not  significantly  different  in  a statisti- 
cal sense.  It  was  concluded  from  the  data  analysis  that  NCLT  training 
seemed  to  assist  experienced  pilots  in  FCLP  by  reason  of  somewhat  higher 
LSO  grades  but  that  the  improved  scores  were  not  statistically  significant. 
Again,  FCLP  periods  5 through  8 showed  the  largest  and  most  consistant 
differentiation  in  RP  landing  performance  as  evaluated  by  LSO  instructors. 

Nugget  Analysis.  The  same  comparison  for  inexperienced  RPs  is 
shown  in  Figure  3.  Note  that  both  groups  of  pilots  start  off  with  lower 
scores  in  periods  1 through  4 than  the  experienced  RPs.  Periods  5 through 
8 again  depict  the  first  indication  of  consistent  differences  in  performance 
between  the  two  groups.  Period  6 performance  was  found  to  represent  a 
significant  difference  between  the  two  groups  at  a p<.  05.  The  overall  mean 
for  all  nine  periods  was  2.  84  for  the  NCLT  Nuggets  and  2.  73  for  the  no- 
NCLT  N\iggets.  The  practical  significance  of  the  data  lies  in  the  consis- 
tently better  performance  for  the  NCLT  Nuggets  in  periods  5 through  8,  just 
prior  to  CQ  trials  aboard  ship. 
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Night  FCLP  Period 

Figure  3.  Comparison  of  Nuggets  (inexperienced  RPs)  with  NCL.T  and  No-NCLT  across 
successive  night  FCLP  periods. 
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Successful  vs  Recycle  Nuggets.  The  last  of  four  figures  which  illus- 
trates LSO  performance  evaluation  of  RP  FCLP  training  progress  describes 
the  successive  night  field  landing  performance  for  sixteen  inexperienced 
RPs  direct  from  the  Training  Command.  Figure  4 shows,  for  the  first  time, 
how  performance  during  FCLP  differs  for  successful  versus  recycle  RPs. 

In  other  words,  the  figure  compares  the  performance  of  low  performers 
(recycle)  with  RPs  who  passed  CQ  on  their  first  attempt  (successful).  None 
of  these  RPs  had  NCLT  ball  training  which  is  now  required.  The  trend  is 
markedly  different  in  this  comparison.  Successful  RPs.  on  the  average, 
scored  consistently  higher  than  unsuccessful  RPs  from  the  onset  of  night 
FCLP  through  period  8.  The  overall  average  for  periods  1 through  8 was 
2.  78  for  successful  and  2.  5 for  recycle  RPs. 

With  the  exception  of  period  5,  FCLP  performance  for  recycle  RPs 
was  quite  stable  at  about  2.  5 for  periods  2 through  8.  Successful  RPs 
improved  their  performance  at  period  5 and  continued  the  same  level  of 
performance  through  period  8.  The  recycle  RPs,  as  a group,  also  improved 
their  performance  at  period  5 but  then  immediately  regressed  at  period  6 
and  continued  the  low  performance  through  period  8,  The  average  FCLP 
performance  for  periods  5 through  8 was: 

Successful  RPs  --  2.  82 
Unsuccessful  RPs  --  2.  55 

Recycle  FCLP  Performance.  A graphic  example  of  night  FCLP  per- 
formance for  three  recycle  pilots  is  shown  in  Figure  5.  Their  individual 
performances  are  compared  to  the  mean  night  FCLP  performance  for  two 
groups  of  successful  and  recycle  Category  I RPs.  Perhaps  the  most  con- 
sistent portrayal  of  their  performance  is  that  of  erratic  and  inconsistent 
grades  across  the  first  six  periods.  Recall  that  the  score  for  each  period 
represents  the  average  of  10  FCLP  landings.  Only  four  scores  for  recycle 
pilots  are  above  the  mean  for  successful  RPs  for  the  first  eight  FCLP 
periods.  The  most  erratic  performance  is  found  in  pilot  number  3 who  was 
disqualified  at  FCLP  and  never  taken  to  the  ship.  His  grades  are  never 
above  the  mean  and  his  sawtooth  performance  signifies  extreme  instability 
in  the  acquisition  of  FCLP  landing  skill.  The  other  two  disquals  are  also 
erratic  but  tend  to  stabilize  and  show  a slight  positive  learning  gradient 
starting  at  period  5. 

Each  recycle  pilot  was  able  to  score  above  2.  6 in  night  FCLP  only 
once  in  the  first  five  periods.  This  is  a remarkable  difference  from 
successful  RPs  who  show  one,  or  at  most  two  grades  below  2.  6 in  the 
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Comparison  of  Night  FCLP  (LSO  Grades)  for  Successful  and  Unsuccessful  RPs 
from  the  Training  Command  (Nuggets). 


Comparison  of  three  disqualified  RP  grades  for  night  FCLP 
with  mean  performance  for  successful  pilots  (Nuggets). 
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first  five  periods.  By  the  time  of  FCLP  pilots  have  had  a considerable 
number  of  A7E  hours  and  many  landings,  both  day  and  night.  Although 
these  are  not  carrier  simulated  landings,  they  do  represent  experience 
in  controlling  and  landing  the  aircraft  in  a day/night  environment.  Erratic 
landing  performance  during  the  first  five  night  NCLT  periods  is  seen  to  be 
a characteristic  of  'recycle'  pilots.  Recycle  pilots  are  more  variable  than 
successful  pilots,  especially  on  a night-to-night  basis.  For  contrast,  a 
single  successful  pilot  has  been  plotted  in  Figure  5 to  graphically  show  the 
reduced  individual  variability  for  successful  pilots. 

FCLP  Cutoff  Scores.  In  an  effort  to  determine  what  FCLP  score  to 
use  as  a criterion  or  cutoff  in  determining  whether  or  not  an  RP  would  be 
identified  as  a potential  recycle  candidate  an  analysis  of  recycle  FCLP 
grades  was  conducted.  Figure  6 shows  the  results  of  the  analysis.  The 
graph  shows  the  percentage  of  actual  grades  at  or  below  each  of  four  FCLP 
grade  levels  for  all  FCLP  periods  (1  through  9)  for  successful  and  recycle 
pilots.  For  example,  using  a criterion  of  2.  8 shows  that  85  percent  of  all 
recycle  pilot  grades  were  at  or  below  2.  8 across  nine  FCLP  periods  while 
57  percent  of  successful  pilots  were  at  or  below  that  grade.  As  can  be  seen, 
the  percentage  progressively  decreases  as  the  scores  decrease  until  at  2.  5 
sixty-four  percent  of  the  recycle  pilots  scored  at  or  below  2.  5 for  all  FCLP 
periods.  Only  17  percent  of  the  successful  pilots  were  graded  that  low. 

The  graph  is  based  on  an  analysis  of  1270  individual  FCLP  landings 
across  16  RPs,  seven  of  whom  recycled  while  nine  were  successful.  The 
particular  use  of  any  of  the  scores  as  a cutoff  is  dependent  on  the  efficiency 
of  any  cutoff  score  in  identifying  a recycle  pilot.  Since  LSO  grading  may 
tend  to  fluctuate,  any  use  of  cutting  scores  should  be  based  on  a current 
analysis  of  FCLP-LSO  grading  trends  much  the  same  as  the  running  aver- 
ages are  kept  for  the  last  50  RPs  across  all  FRS  training  areas.  That  way 
the  scores  would  be  more  representative  of  the  most  recent  grading  trends 
and  averages  for  the  latest  group  of  RPs. 
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Figure  6.  Percent  of  Night  FCLP  Scores  at  or  Below  Three 
Cutting  Scores  for  Recycle  and  Successful  Novice 
Pilots. 
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FCLP  Summary 

The  following  statements  summarize  what  is  known  about  night  FCLP 
training. 

1.  FCLP  grades  can  be  used  to  identify  potential  recycle  pilots 
earlier  in  the  traixiing  sequence. 

2.  Recycle  pilots  tend  to  have  erratic  and  unstable  performance 
(usually  graded  below  2.  6)  throughout  night  FCLP  but 
especially  in  the  first  five  periods. 

3.  Recycle  pilots  are  most  often  direct  from  the  training 
command  (Nuggets)  and  have  substantially  fewer  flight 
hours  (330)  than  other  Category  I trainees  who  have 
three  to  five  times  as  many  flight  hours  and  a much 
lower  recycle  rate. 

4.  Successful  pilots  demonstrate  more  stable  performance 
and  higher  landing  scores  than  recycle  pilots,  especially 
early  in  FCLP. 

5.  FCLP  performance  tends  to  improve  mid-way  through 
FCLP  for  all  Category  I RPs  and  remains  generally  above 
2.  8 for  successful  RPs  for  the  entire  last  half  of  FCLP. 
Unsuccessful  RPs  tend  to  regress  to  lower  performance 
in  periods  six  to  eight. 

6.  A substantial  amount  of  administrative  paperwork  is 
required  of  LSOs  to  compile  the  necessary  performance 
records  for  FCLP.  All  of  the  work  is  manually  perfor- 
med. 

In  summary,  recycle  pilots  can  be  identified  early  in  night  FCLP 
training  by  their  consistent  below  average  performance  (2.  6)  in  the  first 
five  FCLP  periods. 

Two  training  questions  addressed  in  this  report  are;  1)  early  identi- 
fication of  potential  recycle  candidates,  and  2)  disclosure  of  their  specific 
landing  deficiencies.  As  for  the  first  question,  FCLP  training,  as  described 
here,  presents  the  first  intensive  opportunity  to  identify  low  performing  RPs 
who  may  be  candidates  for  recycle.  In  terms  of  the  second  question,  FCLP 
performance,  as  described  by  LSO  log  entries,  provides  an  efficient  and 
realistic  way  to  determine  the  specific  landing  problems  associated  with 
each  RP. 
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SECTION  IV 

FCLP  DIAGNOSTIC  TRAINING  FEEDBACK 


The  data  just  reviewed  represents  a performance  analysis  vdiich  was 
useful  in  isolating  night  FCLP  periods  which  tend  to  reflect  below  average 
performance  by  recycle  pilots.  Specific  individual  landing  deficiencies, 
however,  are  difficult  to  identify  with  summary  statistics  or  grades.  There- 
fore, LSO  comments  on  each  landing  as  recorded  on  FCLP  grade  forms  were 
used  to  obtain  diagnostic  training  information. 

FCLP  Grade  Form.  An  example  of  a FCLP  grade  form  is  shown  in 
Figure  7.  This  form  is  a raw  data  sheet.  The  LSO  logs  and  grades  each 
pass  and  then  averages  them  into  a composite  score  for  each  FCLP  period. 

To  a seasoned  LSO  the  raw  data  sheet  is  a shorthand  record  of  FCLP  per- 
formance which  he  can  integrate  into  a comprehensive  analysis  of  landing 
problems  specific  to  each  replacement  pilot  trainee.  While  the  LSO  can 
accomplish  this  task  by  mentally  reviewing  the  shorthand  symbols  and  com- 
piling a succinct  summary  statement,  an  actual  written  summary  of  landing 
deficiencies  for  each  RP,  for  every  FCLP  period,  is  not  provided. 

Take  the  form  shown  in  Figure  7 for  example.  What  are  the  landing 
deficiencies  for  this  pilot  trainee?  What  diagnostic  training  feedback  could 
be  given  to  him  on  his  overall  landing  technique?  What  are  his  strengths? 
What  are  his  weaknesses?  What  should  he  work  on  to  improve  his  technique? 
What  should  be  stressed  in  his  next  NCLT  training?  Except  to  a seasoned 
LSO  those  questions  cannot  be  answered  from  the  FCLP  landing  form.  The 
raw  data,  however,  are  amenable  to  diagnostic  interpretation  and  a summary 
statement  of  each  FCLP  grade  form  can  be  provided  by  the  LSO  staff. 

A diagnostic  FCLP  summary  statement  of  landing  deficiencies  would 
be  useful  for  several  reasons.  For  instance,  any  interested  party,  including 
LSOs  and  RPs,  could  get  quick  feedback  on  landing  capabilities  and  problems 
without  first  translating  and  interpreting  the  shorthand.  Second,  a diagnostic 
feedback  summary  could  serve  as  a reference  for  progress  in  FCLP  and  be 
compared  with  each  succeeding  FCLP  summary  to  determine  RP  progress 
in  improving  his  landing  technique.  Third,  and  most  important,  a diagnostic 
summary  could  be  used  as  a basis  for  NCLT  remedial  training  to  correct  any 
deficiencies  noted  by  LSOs  during  FCLP.  This  would  be  especially  useful  in 
early  remedial  training  for  low  performers  who  are  prime  candidates  for 
recycle  training. 
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Carrier  Landing  Trend  Analysis.  Fortunately,  there  is  a standard- 
ized fleet-wide  OPNAV  form  that  can  be  used  for  the  purpose  of  presenting 
FCLP  diagnostic  feedback.  It  is  called  the  Carrier  Landing  Trend  Analysis 
Form  and  is  shown  in  Figure  8.  The  form  is  used  primarily  by  LSOs  of 
deployed  units  to  record  pilot  landing  performance  over  time.  The  purpose 
of  the  trend  analysis  record  is  to  provide  a running  account  of  pilot  perfor- 
mance for  review  and  to  notify  pilots  of  any  dangerous  trends.  It  can  be 
used  for  identical  reasons  for  FCLP  performance. 

Basically,  the  form  would  be  used  to  summarize  RP  FCLP  performance. 
It  would  provide  a landing  performance  record  for  each  FCLP  period.  The 
form  has  been  revised  slightly  for  this  purpose.  The  recording  of  wires, 
obviously,  is  not  needed  for  FCLP  since  all  landings  are  touch  and  go  on  the 
field,  so  that  was  deleted  in  the  example.  The  OPNAV  form  could  be  used 
as  is,  however,  without  any  detraction  from  its  purpose  of  providing  feed- 
back for  LSO  and  RP  review  and  to  notify  both  parties  of  and  landing 
technique  deficiencies  or  dangerous  trends. 

The  form  has  been  validated  through  fleet  use  and  is  familiar  to  and 
accepted  by  most  LSOs.  The  content  of  the  form  is  essential  to  its  purpose. 
Namely,  the  columns  are  categorized  by  the  prevalent  landing  technique 
factors  that  are  considered  necessary  for  successful  touchdown.  For  our 
purposes,  the  form  provides  an  off-the-shelf  technique  to  classify  landing 
deficiencies.  The  categories  provide  the  basic  information  needed  to  pro- 
duce a diagnostic  error  analysis  for  each  final  approach  and  landing.  The 
categories  are; 


. Glideslope  Errors 
. Speed  Errors 
. Power  Control  Errors 
. Attitude  Errors 
. Lineup  and  Wings 
. Remarks  (special  comments) 

The  abbreviations  under  Glideslope  and  Speed  Errors  are  short  for  various 
stages  of  approach.  They  are: 


• 

Symbol 

Meaning 

AW 

All  the  way 

OT 

Out  of  turn 

X 

Start 

IM 

In  middle 

IC 

In  close 

f 

AR 

At  ramp 
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Figure  8.  Example  of  Carrier  Landing  Trend  Analysis  form. 
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These  information  categories  form  the  basis  for  specific  diagnostic  training 
feedback  for  different  parts  of  the  approach  to  the  RP  and  LSO.  The  com- 
pleted form  is  shown  in  Figure  9 for  the  same  night  FCLP  data  that  was 
described  in  the  FCLP  grade  form  (Figure  7).  As  illustrated,  the  LSO 
comments  are  rearranged  into  specific  carrier  landing  technique  categories 
and  make  interpretation  of  the  FCLP  data  easier  for  all  interested  parties. 

In  the  example,  the  RP  shows  the  largest  number  of  Glideslope  and  Speed 
Error  comments  in  the  final  stages  of  approach  (in  the  middle,  in  close  and 
at  the  ramp).  It  also  shows  a number  of  power  control  errors,  mostly  in 
close.  No  errors  were  recorded  for  attitude  or  lineup. 

In  a diagnostic  sense,  the  information  described  in  the  Trend  Analysis 
Form  indicates  that  the  RP  does  not  control  his  power  adequately.  He  tends 
to  start  high  and  take  off  power,  but  does  not  put  the  power  back  on  again 
soon  enough  to  stop  from  going  through  the  glideslope  which  catches  up  with 
him  in  close.  He  either  ends  up  underpowered,  which  accounts  for  the  NEP 
(not  enough  power)  and  CD  (come  down)  comments,  or  flat  which  accounts 
for  his  bolters.  This  RP  would  benefit  from  power  management  training  to 
improve  his  glideslope  control.  He  needs  training  in  how  to  anticipate  power 
corrections  while  flying  the  glideslope. 
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Example  of  FCLP  Landing  Trend  Analysis  form  and  entries  for  ten  night  passes 
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SECTION  V 

nclt  remedial  training 


The  analysis  of  FCLP  performance  data  provides  the  basis  to  identify 
potential  recycle  pilots  and  the  specific  landing  problems  that  lead  to  their 
recycle  training.  The  next  step  in  the  proposed  training  concept  is  to  deter- 
mine whether  landing  problems  specific  to  recycle  pilots  can  be  corrected  by 
any  means  besides  additional  recycle  training. 

The  technique  proposed  is  that  of  NCLT  remedial  training.  Since  the 
NCLT  is  a trainer  with  vast  training  capabilities  in  the  area  of  night  landing 
it  was  considered  an  appropriate  vehicle  to  provide  individualized  remedial 
training  to  potential  recycle  pilots.  Because  NCLT  training  is  usually  inter- 
spersed with  FCLP  it  could  be  used  to  provide  immediate  and  specific  train- 
ing to  correct  landing  technique  deficiencies  that  were  noted  in  the  FCLP 
Landing  Trend  Analysis  Form. 

At  present,  the  specific  use  of  the  NCLT  to  provide  the  remedial  train- 
ing is  best  left  up  to  an  experienced  LSO  who  is  qualified  to  provide  NCLT 
training.  The  training  could  be  provided  in  several  ways.  First,  it  might 
be  given  only  to  Nuggets,  relatively  inexperienced  RPs  direct  from  the 
training  command.  Second,  it  might  be  given  to  RPs  anytime  they  score 
below  a given  criterion  (FCLP  score)  for  any  chosen  FCLP  period.  Third, 
it  could  be  given  to  all  Category  I RPs  routinely  as  part  of  individualizing 
the  NCLT  syllabus.  Fourth,  it  could  be  programmed  for  any  RP  who  has  a 
certain  number  of  FCLP  periods  below  a selected  score.  For  example, 
anytime  an  RP  reached  three  grades  below  a cutoff  score  (say  2,  6)  he  would 
be  programmed  to  receive  NCLT  remedial  training  based  on  FCLP  diagnostic 
feedback.  Finally,  the  remedial  training  could  be  used  in  reverse  fashion 
by  reducing  the  FCLP/NCLTs  for  high  performers  (much  like  Category  II 
training  now)  and  using  that  time  to  provide  extra  training  to  potential  recycle 
pilots.  In  this  instance,  an  FCLP  criterion  score  at  the  high  end  could  be 
set  to  identify  acceptable  and  required  FCLP  performance.  For  any  of  the 
above  implimentations  the  LSO  would  remain  the  final  authority  on  RP 
landing  readiness. 

Proposed  Training  Sequence.  A block  diagram  sequence  of  how  the 
FCLP  diagnostic  training  feedback  and  NCLT  remedial  training  would  be 
incorporated  into  the  present  FRS  instruction  is  shown  in  Figure  10.  Train- 
ing time  remains  the  same  depending  upon  which  of  the  alternatives  for 
Category  I training  are  followed. 
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Current  and  Proposed  FRS  Carrier  Landing  Training. 
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Eventually  it  may  be  possible  to  identify  a number  of  landing  deficienci 
that  are  common  to  most  Category  I pilots  due  to  the  specific  landing  profile, 
maneuverability  and  control  characteristics  of  the  A7E  aircraft.  There  ar  * 
already  some  indications  that  that  is  the  case.  If  so,  a complete  revision  ol 
the  NCLT  syllabus  to  emphasize  those  problem  areas  might  have  a direct 
operational  payoff  of  lower  recycle  rates  and  better  landing  performance. 
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SECTION  VI 

FIELD  APPLICATION  AND  TEST 


A field  test  of  the  FCLP  diagnostic  technique  and  NCLT  remedial 
training  was  undertaken  to  judge  and  evaluate  the  potential  of  these  two 
training  concepts.  With  the  cooperation  of  VA-174  personnel  the  training 
concepts  were  applied  and  tested  for  one  class  of  Category  I pilots  during 
mid  1977. 

Four  research  questions  required  answers  to  substantiate  and  sup- 
port the  proposed  training  changes.  They  were: 

1.  Can  potential  recycle  pilots  be  identified  early? 

2.  Can  specific  landing  problems  that  lead  to  recycle 
training  be  identified? 

3.  Are  landing  problems  correctable  without  recycle? 

4.  Does  CQ  performance  improve? 

Method 


Sample.  The  small  sample  chosen  to  test  the  feasibility  of  the 
approach  and  provide  preliminary  answers  to  the  questions  consisted  of  one 
class  of  six  Category  I RPs  and  five  Category  II  pilots  who  were  used  as 
partial  controls.  Of  the  six  Category  Is,  four  were  direct  from  the  training 
command  (Nuggets),  one  was  a jet  instructor  pilot  (plowback),  and  one  was 
previously  assigned  to  a billet  with  variable  flying  time  (stash)  prior  to  FRS 
training.  The  sample  composition  closely  resembled  training  classes  from 
the  NCLT  study  and  was  found  to  be  representative  of  a typical  FRS  training 
class.  The  four  inexperienced  (Nugget)  pilots  averaged  354  jet  flight  hours 
and  the  two  experienced  pilots  (plowback  and  stash)  averaged  1,  703  hours  in 
various  jet  aircraft. 

Performance  Measures.  The  criterion  selected  to  identify  potential 
recycle  candidates  in  this  study  was  a LSO  grade  of  2.  6 or  below  for  any 
three  of  the  first  five  night  FCLP  periods.  This  criterion  was  chosen  in 
conjunction  with  senior  LSOs  to  minimize  false  negatives  and  was  based  on 
an  analysis  of  previous  recycle  pilot  trainee  performance.  In  addition,  it 
was  decided  to  give  the  NCLT  remedial  training  to  all  Category  I RPs 
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regardless  of  flight  hours  for  the  duration  of  training.  In  other  words,  each 
FCLP  period  was  used  to  provide  diagnostic  feedback  and  remedial  NCLT 
training  for  each  Category  I trainee. 

Three  performance  measures  were  used  to  evaluate  the  influence  of 
the  training  techniques  on  performance  effectiveness.  The  primary  measure 
was  the  Landing  Performance  Score  (LPS),  an  objective  method  used  to 
evaluate  carrier  landings.  The  LPS  was  calculated  for  all  day  and  night 
CQ  performance.  A second  quantitative  measure  was  day  and  night  boarding 
rate.  LSO  qualitative  scores  for  FCLP  and  CQ  performance  was  the  third 
measure  used  in  the  evaluation. 

Schedule  of  Training.  NCLT  training  was  interspersed  with  FCLP 
training  during  preparation  for  CQ  trials.  NCLT  training  began  about  three 
weeks  prior  to  ship  CQ  and  one  week  before  FCLP  workups.  The  exact 
integration  of  the  NCLT/ FCLP  training  is  shown  in  Table  7.  The  key  per- 
iods were  weeks  two  and  three  when  NCLT  sessions  followed  FCLP  night 
training  in  order  to  provide  individualized  training  feedback. 

The  amount  of  landing  practice  by  each  training  category  is  shown  in 
Table  8.  NCLT  landings  were  30  percent  less  than  the  standard  syllabus 
in  this  class  due  to  NCLT  trainer  malfunctions.  In  fact,  some 
of  the  NCLT  approaches  were  flown  without  aircraft  motion  due  to  a trainer 
malfunction.  FCLP  practice  and  CQ  landings  were  quite  close  to  the  curric- 
ulum requirements.  FCLP  landings  were  about  10  percent  less  than  the 
curriculum  requirement  while  CQ  trials  were  very  close  to  the  required 
number. 

TABLE  8.  AVERAGE  NUMBER  OF  LANDINGS  DURING  TRAINING 


Actual 

Curriculum 

Training 

Day 

Night 

Required 

NCLT 

56 

80 

FCLP 

36.  7 

83 

40  & 90 

CQ 

10.  1 

7.2 

10  & 6 

Brictson,  C.A.  Validation  and  Application  of  a Carrier  Landing  Perfor- 
mance Score:  The  LPS.  Dunlap  and  Associates,  Inc.  March,  1973. 

See  the  Appendix  for  further  discussion  of  LPS. 
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TABLE  7.  SCHEDULE  OF  NCLT  AND  FCLP  TRAINING 
FOR  SIX  CATEGORY  I RPs 

Week  1 M T W T F S 


NCLT 

- 

- 

1 

- 

- 

FCLP 

- 

- 

- 

- 

Week  2 

Remedial  Training 

NCLl 

- 

2 

3 4 

- 

FCLP 

ID/IN 

- 

2N 

2D/3N 

- 

Week  3 

Remedial  Training 

NCLT 

5 

- 

6 7 

- 

FCLP 

4N 

5N 

6N  7N 

3D/8N 

9N 

Week  4 

NCLT 

8 

- 

FCLP 

4D/10N 

Ship  CQ  USS  Saratoga 


Note:  D = Day,  N = Night,  1 = 1st  training  period, 
e.  g.  3D  = 3rd  Day  FCLP  period. 
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Procedure.  Normal  syllabus-related  Phase  III  instruction  was 
followed  throughout  the  class  training  with  the  addition  of  individualized 
diagnostic  feedback  and  remedial  NCLT  for  each  night  FCLP  period  by 
the  LSOs.  These  summaries  were  developed  from  the  raw  FCLP  Data 
Forms  and  used  to  identify  specific  landing  problems  for  each  RP  during 
every  FCLP  period.  In  addition,  for  each  diagnostic  summary,  LSOs  pre- 
scribed a remedial  action  designed  to  use  the  NCLT  capabilities  to  correct 
or  provide  training  in  the  specific  problem  areas.  Remedial  NCLT  train- 
ing periods  followed  most  night  FCLP  sessions  (usually  the  same  day)  in 
order  to  provide  the  individualized  training.  Thus,  a written  diagnostic 
FCLP  summary  and  a recommended  remedial  action  was  available  at  the 
very  next  NCLT  period  for  each  RP  after  every  FCLP  period.  This 
required  a considerable  amount  of  extra  paper  work  for  the  LSOs  involved. 
A normal  night  FCLP  period  would  end  anywhere  from  midnight  to  2 a.m. 
after  which  debriefs  of  at  least  one  hour  were  held.  Sometime  between 
then  and  the  following  afternoon  LSOs  had  to  transcribe  the  FCLP  data 
from  their  log  book  to  the  FCLP  Data  Forms,  then  to  the  Trend  Analysis 
Forms,  write  a summary  and  prescribe  a remedial  action  for  each  RP. 

Recycle  Predictions.  After  the  fifth  night  FCLP  period  an  indepen- 
dent (non-LSO)evaluation  of  all  RPs  was  conducted  and,  using  the  2.  6 or 
below  grading  criterion,  predictions  were  made  to  identify  potential  recycle 
candidates  based  on  their  FCLP  performance.  On  the  basis  of  that  evalua- 
tion two  RPs  were  predicted  as  recycle  candidates.  One  had  three  FCLP 
grades  below  the  2.6  criterion  while  the  other  had  four  FCLP  grades  below 
2.  6 after  five  night  periods.  Based  on  an  analysis  of  53  previous  RPs  from 
the  NCLT  study  neither  of  these  RPs  was  expected  to  pass  CQ  training 
based  on  their  FCLP  performance  in  the  first  five  FCLP  periods. 


Results 


The  results  of  the  field  test  are  shown  in  several  tables  and  graphs 
which  illustrate  the  average  performance  of  selected  RPs  across  FCLP  and 
CQ  training  periods.  Three  sets  of  performance  data  are  provided  for 
contrast  and  comparison. 

. NCLT  - Remedial  Group  (individualized  training  for  Cat  Is) 

. NCLT  - Normal  Group  (regular  NCLT  syllabus  given  to  Cat  Is) 

. NCLT  - Experienced  Group  (regular  NCLT  given  to  RPs  with 
over  1,000  flight  hours) 


43 


NAVTRAEQUIPCEN  75-C-0105-1 


Data  for  the  NCLT  normal  and  experienced  groups  are  taken  from  the  NCLT 
study  for  comparison  purpose. 

Performance.  Table  9 and  Figure  11  show  the  results  of  the  field  test. 
For  the  two  primary  objective  measures  - LPS  and  Boarding  Rate  - the 
remedial  group  scored  higher  in  both  measures,  day  and  night,  compared  to 
the  Normal  and  Experienced  NCLT  Groups. 

Remedial  night  LPS  was  higher  (4.  63)  than  the  NCLT -Normal  and 
NCLT-Experienced  Groups  (4.27)  as  was  their  day  LPS  (5.  19  vs  4.  52  and 
4.81,  respectively).  The  most  significant  finding  was  in  night  and  day 
boarding  rate  which  was  substantially  higher  for  the  Remedial  Group.  At 
night  their  boarding  rate  was  21  percent  higher  (83  percent)  than  their  Nugget 
counterparts  (62  percent)  who  had  a normal  NCLT  course  and  also  slightly 
higher  (83  percent  vs  79  percent)  than  that  for  experienced  Category  I pilots. 
Further  evidence  of  high  performance  was  the  fact  that  the  CQ  period 
required  only  two  days  and  nights,  an  absolute  minimum  time  to  qualify 
aboard  ship. 

LSO  qualitative  grading  of  day  and  night  approaches  supported  the 
same  higher  performance  trend.  Remedial  pilots  were  graded  higher  in  day 
and  night  CQ  by  the  training  LSOs  and  were  judged  to  be  above  the  mean  per- 
formance of  the  last  50  RPs  who  graduated. 

FCLP  LSO  grades  were  only  slightly  higher  for  the  Remedial  Group, 
day  (3.  0 vs  2.  88)  and  night  (2.  9 vs  2.  84).  Compared  to  experienced  RPs, 
day  performance  was  somewhat  higher  for  the  Remedial  Group  (3.  0 vs  2.  88) 
and  about  the  same  at  night  (2.  9 vs  2.  92). 

Recycle  Prediction  Results.  Two  RPs  were  predicted  as  recycle 
pilots.  The  results  confirmed  one  prediction  (4  of  5 FCLPs  below  2.  6)  and 
were  false  on  a second  (3  of  5 grades  below  2.  6)  who  passed  CQ  on  his  initial 
attempt.  The  recycle  ratio  was  1/6  for  the  Remedial  Group,  1/13  for  the 
Normals  and  0/13  for  the  Experienced  Group. 

Night  FCLP  Performance.  The  results  of  the  FCLP  performance 
analysis  shows  an  overall  night  average  of  2.  9 for  the  five  successful  RPs 
and  2.4  for  the  one  recycle  pilot.  A plot  of  the  comparative  performance 
is  shown  in  Figure  12.  Successful  pilots  show  a rather  consistent  and 
gradual  learning  curve  that  starts  at  about  2.  6 and  reaches  an  average  of 
3.  0 for  the  last  three  FCLP  period.  Actually,  except  for  period  7,  six  out 
of  the  last  seven  periods  show  a consistent  performance  level  of  3.  0. 
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TABLE  9.  COMPARISON  OF  NUGGET  AND  EXPERIENCED  RP  CQ 
PERFORMANCE  FOR  TWO  TYPES  OF  NCLT  TRAINING 


Nuggets 

Experienced  RPs 

NCLT  - Remedial 
N=6 

NCLT  - Normal 
N=14 

NCLT  - Normal 
N=13 

Day  FCLP 

3.0 

2.88 

2.88 

Night  FCLP 

2.9 

2.84 

2.  92 

Day  CQ  (LSO) 

2.  90* 

2.82 

2.  86 

Night  CQ  (LSO) 

2.  91* 

2.  75 

2.  7"’ 

Day  CQ  (LPS) 

5.  19* 

4.  52 

4.81 

Night  CQ  (LPS) 

4.63* 

4.27 

4.  27 

Boarding  Rate 

Day 

98% 

88% 

91% 

Night 

83.  3% 

62% 

79% 

Recycle  Ratio 

1/6 

1/13 

0/13 

Recycle  % 

16% 

8% 

0 

* - N=5 

( 
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Summary  Comparison  of  Nugget  and  Experienced  RPs  for  Two  Tvpes  of 
NCLT  Training 


Comparison  of  Successful  vs  Recycle  Night  FCLP  Performance  for  Category  1 
(Nugget)  Pilots. 
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The  recycle  RP  performance  is  characterized  by  the  expected  erratic 
performance  with  no  consistent  learning  curve  or  gradual  improvement.  As 
can  be  seen,  four  of  his  first  five  night  FCLP  periods  were  graded  below 
2.  6.  In  fact,  only  three  of  his  10  FCLPs  were  above  2.  6. 

The  FCLP  grades  for  the  false  negative  RP  prediction  (3  of  5 FCLPs 
below  2.  6)  are  shown  in  Figure  13.  Three  of  his  first  five  FCLPs  are  at 
or  below  2.  6,  but  only  one  of  the  last  five  are  below  that  cutoff  criterion 
score.  Out  of  nine  total  FCLPs,  five  are  above  2.6.  Again,  notice  the 
erratic  FCLP  performance  scores.  Especially  pertinent  to  this  false  pre- 
diction are  the  LSO's  summary  comments. 

"Ensign successfully  day/night  carrier  qualified  aboard 

USS  Saratoga  under  good  weather  conditions.  His  grades  are  not 
a true  indicator  of  his  skill  as  an  aviator.  Initially  he  was  some- 
what tense  and  as  a result  over  controlled  the  aircraft.  His  field 
performance  was  strong  and  as  he  becomes  familiar  with  the 
carrier  environment  he  will  relax  and  rapidly  improve.  He  will 
be  a major  asset." 

As  for  his  CQ  work,  his  day  boarding  rate  was  90  percent  and  his  night 
performance  consisted  of  six  arrestments  and  three  bolters  for  a boarding 
rate  of  67  percent.  His  night  CQ  work  as  graded  by  the  LSO  was  2.  3 com- 
pared to  a class  average  of  2.  94. 

CQ  Performance  Comparison.  How  well  the  remedial  pilots  scored 
during  night  and  day  CQ  trials  can  be  evaluated  by  comparison  to  carrier 
landing  performance  scores  for  other  A7  samples.  Figure  14  shows  that 
comparison.  The  two  distributions  are  based  on  over  25,  000  carrier 
landings  for  seventeen  ship/aircraft  combinations. 

A7  remedial  pilots  scored  4.  63  on  the  LPS  scale  which  places  them 
considerably  above  other  A7  Nugget  pilot  scores  for  both  night  and  day.  A 
night  CQ  score  of  4.  63  is  equivalent  to  a percentile  score  of  . 62  for  all 
night  carrier  landing.  That  means  that  only  28  percent  of  all  night  carrier 
landing  LPS  scores  are  better  than  the  remedial  Nugget  score.  In  fact,  the 
day  CQ  performance  was  higher  than  a combat  deployed  squadron  of  A7 
pilots  and  represents  a 90th  percentile  for  all  day  carrier  landings. 

The  LPS  comparison  shows  a clear  performance  superiority  for 
remedial  NCLT  pilot  performance  compared  with  all  other  types  of  CQ 
training  groups. 
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'igure  14.  Comparison  of  A7  performance  across  day  and  night  normative  LPS  Hi striliutions 
for  25.000  landings  across  seventeen  ship/aircraft  combinations. 
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Research  Answers 


The  field  test  described  here  was  designed  to  provide  preliminary 
verification  and  support  for  two  training  concepts  designed  to  reduce  the 
impact  of  FRS  recycle  training  for  Category  I pilot  trainees.  Answers 
obtained  to  the  four  basic  research  questions  follow. 

Question  # 1 : Can  potential  recycle  pilots  be  identified  early? 

Answer:  Potential  recycle  pilots  can  be  identified  by  the  fifth  night 
FCLP  period  by  selecting  an  FCLP  cutting  score  and  determining  if  the  RP 
is  either  erratic  in  performance  or  below  the  cutting  score  in  three  or  four 
of  the  first  five  night  FCLP  periods. 

Question  #2:  Can  specific  landing  problems  that  lead  to  recycle 
training  be  identified? 

Answer:  An  analysis  of  FCLP  LSO  comments  can  be  used  to  evaluate 
each  RP  landing  technique  and  provide  diagnostic  feedback  on  landing 
deficiencies  using  a landing  trend  analysis  form  to  summarize  the  data. 

Question  #3:  Are  landing  problems  correctable  without  recycle? 

Answer:  This  question  requires  additional  study  before  a firm  answer 
is  stated.  In  this  study  the  landing  problems  of  one  RP  who  was  a low  per- 
former were  sufficiently  corrected  using  NCLT  remedial  training  to  allow 
him  to  pass  CQ  on  his  initial  try  and  avoid  recycle  training.  A second  pre- 
dicted recycle  RP  was  not  salvaged  by  remedial  training  and  was  recycled 
through  a complete  second  training  cycle  for  CQ. 

Question  #4:  Does  CQ  performance  improve? 

Answer:  The  results  clearly  indicate  an  improvement  in  Nugget  CQ 
performance  compared  with  non-remedial  trainees.  An  exceptionally  high 
night  boarding  rate  of  83  percent  was  obtained  compared  to  an  average  night 
boarding  rate  of  62  percent.  Whether  the  improved  performance  is  sus- 
tained in  the  fleet  should  be  followed-up  in  a longitudinal  study  of  all 
Category  I {ulots. 


( 
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SECTION  VII 

RECYCLE  PILOT  FLEET  PERFORMANCE 


Eight  recycled  Category  I pilots  were  the  subjects  of  a fleet  follow-up 
study  to  determine  how  well  they  performed  in  a fleet  squadron.  For  the 
covered  period  nine  pilots  were  recycled  through  Phase  III  CQ  training  but 
data  were  obtained  on  only  eight  of  that  group.  Of  those  eight  pilots,  one 
received  the  full  NCLT  syllabus  and  seven  did  not  receive  any  NCLT  train- 
ing in  their  first  CQ  cycle.  The  NCLT  trainee  was  the  only  pilot  to  disqual 
because  of  day  performance  and  was  the  only  pilot  who  received  three  train- 
ing cycles.  He  is  identified  as  pilot  #2  in  Table  10. 


Table  10.  DAY  AND  NIGHT  FCLP  GRADES  FOR  RECYCLED 
category  I PILOTS 


Day  FCLP  Night  FCLP 


Pilot 

1st  Cycle 

1st  Cycle 

Recycle 

1 

2.23 

2.  22 

2.40) 

2 

2.  55 

2.  67 

DQ 

3 

2.  15 

2.42 

2.  56) 

4 

2.  60 

2.  67 

2.  64 

5 

3.  05 

2.  73 

2.  50 

6 

3.  07 

2.  35 

2.40 

7 

2.  55 

2.  70 

2.  94 

8 

2.  71 

2.  71 

3.  02 

Avg.  2.  62 

2.  56 

2.  64 

Given  Pilots 
Board  in  Fleet 


Night  FCLP  performance  improved  only  slightly  during  the  recycle 
session.  The  recycle  pilots  averaged  2.  56  and  2.  64^  respectively,  in  their 
first  and  second  night  FCLP  training.  Recycle  day  performance  (2.  62)  was 
also  lower  on  the  average  than  successful  pilots.  The  average  grade  for 
successful  pilots  was  2.82  day  and  2.  75  night. 

Fleet  performance  of  each  recycled  trainee  was  obtained  and  is 
presented  in  Table  11. 
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TABLE  11.  RECYCLE  PILOT  FLEET  PERFORMANCE 

Pilot 

Fle*it  Performance 

1 

Given  pilots  board  and  disqualed  from  his  fleet  squadron. 
Last  at  sea  period  his  boarding  rate  was  35%.  Major 
difficulty  was  day  pattern.  Slightly  better  at  night  but 
was  getting  worse.  Made  improper  corrections. 

2 

Minor  ramp  strike  and  was  boarded  out  after  105  traps. 

Was  hot  and  cold. 

3 

Boarded  for  poor  landings.  Fleet  training  limited  and 
performance  unsatisfactory.  Returned  to  FRS  for  NCLT/ 
FCLP  full  syllabus. 

4 

Started  out  poorly  but  has  progressed.  Last  at  sea  period 
85%  boarding  rate.  Total  traps  to  date,  48  day  and  18 
night  - average  grades. 

5 

Last  at  sea  period,  6 day  and  4 night,  3.  55  grade.  No 
problems  at  present  time. 

6 

No  problems;  doing  well. 

7 

No  problems.  LSO  says  he  is  their  bad  weather  pilot. 

8 

Doing  good  - ranks  in  top  third  of  squadron. 

C 
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Three  of  the  eight  recycled  pilots  were  unsatisfactory  in  their  fleet 
performance.  All  three  received  pilot  boards  and  there  was  one  aircraft 
accident  (ramp  strike).  The  fleet  failure  rate  for  recycled  pilots  was  37.  5 
percent  for  the  sample  of  A7E  Category  I trainees. 

Pilots  #1,  #2  and  #3  received  the  fleet  disqualifications.  Their  FCLP 
grades  during  recycle  were  all  below  the  mean  for  all  recycle  pilots  (2.  64). 
Pilot  #2,  the  lone  NCLT  trainee,  received  three  training  cycles.  He  was 
disqualed  for  day  CQ  performance  in  his  first  cycle,  passed  night  CQ  but 
again  failed  day  CQ  in  his  second  cycle  and  passed  on  his  third  cycle.  He 
was  also  the  lone  pilot  to  have  an  accident. 

An  analysis  of  FCLP  grades  for  the  three  fleet  disquals  was  perfor- 
med to  see  if  additional  information  could  be  obtained  on  the  relation  between 
early  FCLP  performance  and  fleet  disqual  performance.  Applying  an  FCLP 
criterion  cutting  score  of  2.  6 the  three  pilots  performed  as  shown  below. 

No.  of  FCLPs  below  2.  6 


Pilot 

Day 

Night 

Night  % 

1 

2 of  3 

6 of  7 

86% 

2 

2 of  3 

6 of  11 

55% 

3 

2 of  2 

8 of  9 

89% 

i 

Pilots  #1  and  #3  had  a high  percentage  of  night  FCLP  scores  below 
the  2.  6 criterion  (86%  and  89%)  and  would  have  been  identified  as  low  per- 
formers using  the  2.  6 cutoff.  Additional  remedial  training  may  have 
salvaged  pilot  #3  since  he  had  to  return  for  what  amounted  to  a 3rd  recycle 
with  his  pilots  board  action.  His  current  fleet  performance  is  not  known. 

Pilot  #2  indeed  ran  "hot  and  cold"  and  his  FCLP  night  performance  showed 
that  he  was  below  criterion  in  his  last  five  night  FCLPs. 

In  terms  of  time  of  FRS  recycle  the  three  fleet  disquals  were  some- 
what vinique.  Two  were  recycled  due  to  night  FCLP  (#1  and  #3)  while  #2 
was  a day  CQ  recycle.  Most  recycles  (50%)  who  succeeded  in  the  fleet  were 
recycled  due  to  poor  night  CQ  performance.  Apparently,  all  of  these  pilots 
mastered  the  FCLP  landing  skill  but  not  the  CQ  skill. 

For  future  reference  it  is  interesting  to  note  that  novice  pilots  who 
recycle  due  to  night  FCLP  or  day  CQ  have  a high  (75%)  fleet  disqualification 
rate  (3  of  4).  On  the  other  hand,  novice  pilots  who  recycled  due  to  night  CQ 
were  successful  in  their  fleet  assignments  in  every  case  (4  of  4)  for  this 
small  sample  of  trainees. 

0 
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Presentation  of  this  information  on  fleet  performance  of  recycled 
pilots  may  prove  useful  to  FRS  COs,  training  officers  and  LSOs  who  are 
responsible  for  decisions  on  recycle  training.  Although  the  equivalent  fleet 
failure  rate  for  successful  Category  I pilots  is  not  known  it  is  suspected  to 
be  much  lower  than  the  37.  5 percent  failure  rate  disclosed  for  recycled 
pilots.  On  the  other  hand,  five  of  the  eight  pilots  were  successful  in  fleet 
performance  and  are  now  considered  fully  qualified  and  acceptable  fleet 
aviators.  What  does  seem  clear,  although  based  on  the  limited  data  of  this 
study,  is  that  three  recycles  are  rare  and  not  very  successful.  Further 
fleet  performance  follow-up  of  recycle  and  other  Category  I trainees  is 
recommended  as  a training  method  to  provide  feedback  and  validation  to 
FRS  personnel  to  improve  training  effectiveness. 
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SECTION  vm 
RECOMMENDATIONS 


1.  The  training  concept  of  night  FCLP  performance  analysis,  diagnostic 
feedback  and  NCLT  remedial  training  should  be  validated  through  a 
more  intensive  field  study  of  A7  pilot  trainees.  In  addition,  the  con- 
cepts should  be  applied  across  other  FRS  squadrons  (F4,  A7,  A6, 

F14)  especially  those  with  visual  landing  simulators  (F14). 

2.  Fleet  performance  follow-up  of  all  recycled  pilots  should  be  started 
and  formalized  with  fleet  squadrons.  All  Category  I trainees  should 
be  tracked  in  their  fleet  performance  in  order  to  provide  operational 
feedback  for  FRS  training  personnel. 

3.  FCLP  performance  analysis  should  be  automated  to  reduce  LSO  ad- 
ministrative workload,  assure  effective  remedial  training  and  provide 
for  eventual  integration  with  the  Variable  Training  System  (VTS)  now 
under  development  for  pilot  readiness  training. 

4.  Further  refinements  in  the  sequence  of  FCLP/NCLT  training,  identi- 
fication of  specific  A7  trainee  landing  problems  and  innovative  reme- 
dial instruction  techniques  to  reduce  recycle  training  and  enhance  fleet 
performance  should  be  investigated. 

5.  Greater  use  of  NCLT  capabilities  as  a trainer  should  be  emphasized. 
These  uses  should  include  more  NCLT  instruction  specific  to  indivi- 
dual pilots  to  include  acquisition  of  basic  landing  skills  and  correction 
of  specific  landing  problems.  Another  use  would  be  to  train  pilots  to 
a criterion  rather  than  a standard  syllabus  which  is  now  given  to  all 
pilots  regardless  of  their  unique  learning  requirements. 

6.  Some  form  of  instructional  systems  technology  should  be  applied  to 
Phase  III  CQ  training  to  emphasize  and  incorporate  the  above  recom- 
mendations. Included  in  this  approach  would  be  examination  of  the 
potential  benefits  of  a positive  CQ  experience  (successful)as  opposed 
to  recycle,  and  the  possibility  of  forwarding  all  FCLP  and  NCLT 
remedial  summary  sheets  to  each  pilot's  new  squadron  for  training 
purposes. 
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APPENDIX  A ' 

THE  LANDING  PERFORMANCE  SCORE  (LPS)  AS  A CRITERION 
THE  LPS^ 

There  are  a number  of  desirable  characteristics  that  our  criterion 
or  LPS  should  exhibit.  It  should  be  objective,  generally  applicable  to  the 
various  system  components,  readily  available  and  easily  applied.  It  must 
be  a valid  and  reliable  measure  of  operational  performance. 

Objectivity  - An  objective  criterion  measure,  uncontaminated  by 
subjective  bias  and  inherent  rater  variability  across  individual  data  collec- 
tors would  be  more  useful  to  system  design  evaluation  than  conventionally 
available  subjective  estimates  of  system  performance. 

Generality  - A criterion  measure  which  is  generally  related  to  various 
system  components  is  most  efficient.  Ideally,  we  would  like  a single  mea- 
sure which  could  serve  to  discriminate,  for  example,  among  performances 
obtained  under  different  ship,  aircraft,  pilot,  LSO  or  environmental  condi- 
tions across  several  training  and  operational  recovery  modes. 

Availability  - Inherent  variability  in  system  equipment,  environment, 
experience,  and  operations  produces  substantial  criterion  variability  with 
the  consequence  that  landing  statistics  need  to  be  based  upon  large  samples. 
A technique  which  requires  unique  measurement  equipment  or  extensive  data 
manipulation  is  less  desirable  than  one  which  requires  only  data  normally 
collected  and  available  as  a matter  of  policy.  Landing  data  now  routinely 
recorded  aboard  ships  are  easily  obtained  and  are  likely  to  be  available  in 
the  quantities  necessary  to  allow  use  of  large  sample  statistics. 

Operational  Validity  - To  be  useful  our  criterion  must  be  operationally 
valid,  that  is,  it  should  measure  and  reflect  actual  landing  performance  and 
be  closely  related  to  aircraft  recovery  criteria  which  have  been  used  in  the 
fleet.  As  mentioned  earlier,  operational  criteria  which  have  been  used 
include  accident  rate,  boarding  rate,  bolter  rate,  LSO  approach  evaluations 
and  final  approach  parameters. 


1 


Extracted  from:  Brictson,  C.A.  et  al.  Validation  and  Application  of  a 
Carrier  Landing  Performance  Score:  The  LPS.  Dunlap  and  Associates,  Inc. 
March  1973. 
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Reliability  - Our  criterion  must  be  reliable  if  it  is  to  be  of  any  use  at 
all.  To  be  reliable  it  must  provide  a stable  and  consistent  measure  of 
similar  performance,  i.  e.  , if  landing  performance  is  high  the  criterion 
will  consistently  register  it  as  being  high.  Some  measure  of  reliability  is 
required  to  reflect  the  consistency  of  performance  over  time  and  the 
stability  of  performance  measured. 

Each  of  the  carrier  landing  criterion  measures  we  have  used  pre- 
viously is  lacking  with  regard  to  one  or  more  of  those  desirable  character- 
istics. In  this  section  we  describe  in  detail  the  LPS  which  has  all  the 
desirable  characteristics  and  in  addition  integrates  several  operational 
criteria  into  a single  landing  performance  score  which  reflects  the  quality 
of  final  approach  and  landing. 

Some  of  our  previous  research  indicates  that  the  longitudinal  touch- 
down point  on  the  carrier  deck,  i.  e.  , wire  arrestment  number,  is  highly 
predictable  from  final  approach  measures  such  as  lateral  and  altitude 
errors  from  glideslope  and  sink  rate.  Sixty-five  percent  of  the  variance 
in  touchdown  point  has  been  accounted  for  by  measures  recorded  during 
final  approach.  Final  approach  variables  have  been  combined  by  multiple 
correlation  techniques  to  predict  the  quality  of  pilot  landing  performance 
during  night  carrier  recovery  when  a linear  regression  model  was  used  to 
predict  landing  quality  from  final  approach  measures.  A multiple  R of  . 81 
was  obtained  for  inexperienced  F4  pilots  and  an  R of  . 71  for  combat  exper- 
ienced A4  pilots.  Both  Rs  were  found  to  be  significant  at  the  p<  .01  level 
and  were  validated  against  independent  samples.  Thus,  touchdown  point  is 
seen  to  be  related  to,  and  can  be  predicted  from,  measures  of  final  approach 
performance.  Consequently,  a criterion  which  integrates  the  degree  or 
quality  of  landing  success  in  terms  of  touchdown  point  (wires)  as  well  as 
the  degree  of  landing  failure  (in  terms  of  bolters  and  technique  waveoffs) 
and  scales  success  and  failure  on  a relative  basis  can  be  used  as  a more 
versatile  objective  measure  of  approach  and  touchdown  performance  than 
is  currently  available. 

Final  approaches  must  result,  with  a few  exceptions  such  as  accidents, 
foul  deck  waveoffs  and  flybys,  in  one  of  the  landing  categories  shown  below. 
By  obtaining  LSO  estimates  of  the  relative  weights  for  each  landing  category 
the  following  criterion  scoring  system  was  devised. 
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The  EPS 


Landing 

Categories 


Rank  LSO  Scaled 

Order  Landing  Score  Intervals 


Wire  3 (target)  1 6.  0 

Wire  2 2 5. 0 

Wire  4 3 4,  5 

Wire  1 4 3. 5 

Bolter  5 2.0 

Own  or  technique  waveoff  6 1.0 


It  should  be  noted  that  in  this  scoring  system,  hook- skips  are  corrected 
for  the  appropriate  wire  number  and  foul-deck  waveoffs  or  other  failures 
not  due  to  performance,  such  as  equipment  malfunctions  or  ship  turns,  are 
excluded. 


The  landing  categories  were  rank  ordered  by  22  senior  LSOs  to  obtain 
an  ordinal  scale  of  landing  quality  ranging  from  a target  wire  arrestment  to 
a technique  wave -off.  The  next  step  in  the  development  of  the  LPS  was  to 
convert  the  rank  order  data  into  an  interval  scale  to  obtain  a single  quanti- 
tative value  for  each  landing  category.  It  was  thought  that  the  scaling  tech- 
nique of  paired  comparisons  could  be  used  to  order  the  variability  among 
raters  and  serve  as  the  basis  for  forming  such  an  interval  scale  of  landing 
quality.  However,  a pre-test  of  this  approach  resulted  in  a lack  of 
variability  and  for  statistical  purposes  an  almost  universal  agreement 
concerning  which  of  each  pair  of  categories  was  superior  in  quality  to  the 
other.  It  was  finally  decided  to  use  the  22  LSOs  as  judges  to  scale  the  land- 
ing outcomes  by  having  them  assign  numbers  to  each  landing  category  on  the 
basis  of  the  relative  value  of  each  outcome  with  respect  to  every  other  out- 
come. The  method  of  successive  categories,  also  called  the  method  of 
successive  intervals,  as  described  by  Guilford  (1954)  was  used  to  derive 
the  interval  scale  measurements,  A five-point  graphic  scale  ranging  from 
1 to  6 and  graduated  in  half  unit  intervals  was  used.  LSOs  were  asked  to 
assign  a value  from  1 to  6 to  each  landing  category.  A value  of  one  was 
used  to  represent  the  poorest  and  six  the  best  landing  outcome  in  their 
judgment.  LSO  categorical  judgments  were  then  scaled  to  obtain  a central 
value  representative  of  each  category.  The  scaling  technique  revealed 
relatively  equal  discriminal  dispersions  between  landing  categories  and  a 
high  degree  of  internal  consistency  in  the  data.  The  final  result  was  an 
equal  interval  scale  of  landing  quality  that  represented  an  LSO  concensus 
of  the  relative  numerical  value  of  each  possible  landing  outcome. 
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The  principal  advantage  of  the  LPS  is  that  it  is  based  upon  data  routinely 
recorded  on  all  CVAs.  The  data  source  is  typically  the  LSO  landing  log  which 
identifies  each  landing  attempt  as  to  time  of  day,  pilot  name,  aircraft  side 
number,  weather,  lighting,  glideslope  and,  most  important,  the  landing  status 
. . . wire  number,  bolter  or  waveoff.  Those  data  are  recorded  routinely  in 
the  same  manner  for  all  carrier  landings,  across  all  CVAs, 

Comparisons  within  the  carrier  landing  system  and  across  its  system 
components  can  be  made  by  transforming  each  pilot/aircraft  approach  into 
a performance  score  and  computing  simple  statistics.  Means  and  variances 
can  then  be  used  in  tests  to  determine  any  statistically  significant  or  practi- 
cal differences  in  the  comparisons. 
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